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Decorative mats plaited by the Iban communities in Borneo contains motifs 
that reflect their traditional beliefs. Each motif has its own special meaning 
and taboos. A typical mat motif contains multiple smaller patterns that 
surround the main motif hence is likely to cause misclassification. We 
introduce a classification framework with adaptive sampling to remove 
smaller features whilst retaining larger (and discriminative) image structures. 
Canny filter and probabilistic hough transform are gradually applied to a 
clean greyscale image until a threshold value pertaining to the image’s 
structural information is reached. Morphological dilation is then applied to 
improve the appearance of the retained edges. The resulting image is 
described using binary robust invariant scalable keypoints (BRISK) features 
with random sample consensus (RANSAC). We reported the classification 
accuracy against six common image deformations at incremental degrees: 
scale+rotation, viewpoint, image blur, joint photographic experts group 
(JPEG) compression, scale and illumination. From our sensitivity analysis, 


we found the optimal threshold for adaptive smoothing to be 75.0%. The 
optimal scheme obtained 100.0% accuracy for JPEG compression, 
illumination, and viewpoint set. Using adaptive smoothing, we achieved an 
average increase in accuracy of 11.0% compared to the baseline. 
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1. INTRODUCTION 

Bormeo is widely known as a home to the world’s richest and aesthetically appealing plait work 
traditions. These traditions derive from the island’s diverse cultural history and a product of creativity as well 
as ingenuity of its different communities [1]. Decorative mat plaiting is a craft skill known in most 
indigenous communities in Borneo. Plaited mats are used for basic floorcovering, for sitting, for sleeping and 
for ritual purposes during religious ceremonies. Sleeping and ritual mats tend to be the most elaborately 
decorated and may feature sacred motifs that are believed to be carrying spiritually powerful patterns that 
establish link between humans and the gods [2]. Modern practice of naming plaited mat motifs has suffered 
from a relative neglect in material culture studies, often resulting to a devoid of meaning. The relation 
between a motif and the name by which it is called is often not relevant. Thus, it is specious to interpret 
motifs only through their names [3]. Our previous work [4] investigated the use of an invariant image-based 
feature descriptor for recognizing Iban plaited mat motifs. We argue that this is the best route (i.e., image- 
based classification) to automate the mat motif recognition task. With modern smartphones being equipped 
with camera(s) as standard, we can integrate feature into educational and edutainment app. Such apps will be 
an excellent way to promote and educate the public about this invaluable cultural heritage to the masses. 
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This paper focuses on plaited mat work by the Iban communities. Plaited mats contain unique motifs 
that are either simple or complex. The motifs are mostly stylized beyond recognition. Most motifs are plaited 
based on loose categories that include natural elements such as plants, animals, firmaments, and faerie [5]. 
The main motif plays a dominant role for determining the specific meaning, whereas the smaller motifs only 
serve as fillers for filling in the empty areas surrounding the main motif. These motifs are usually arranged 
repeatedly to illustrate the mat pattern as a whole see Figure 1. Each motif has its own distinctive features. 
Thus, to recognize a plaited mat motif, we need to extract discriminative as well as robust visual features. 
Nevertheless, due to the diversity of appearances, variety of positions, scales and rotations of the major and 
minor motifs, the recognition task is non-trivial. Furthermore, the image captured by the camera may undergo 
common image deformations hence making the task harder. 


Figure 1. An example of an Iban plaited mat motif, i.e., Buah Tungku. The (1) and (2); small ornament that 
act as a filler, Buah Mata Punai. The image was sourced from [4] 


To date, a significant amount of research had been done on the robust detection, description and 
matching of invariant features related to motif and pattern classification. Features extraction algorithm and 
classification methods were applied to batik motif and batik making using convolutional neural network 
(CNN) model architectures [6]-[10], using multiwindow and multiscale extended center symmetric local 
binary patterns (MU2ECS-LBP) [11] and [12], using scale invariant feature transform (SIFT) [13], [14] using 
gray level co-occurrence matrices (GLCM) [15], [16], fine arts [17]-[19], for license plate recognition [20], 
[21], and tattoo recognition [22]—[24]. Even though the reported performance was quite high but these 
methods still suffer from false positives due to similar features that contain more than one pattern and noisy 
background. In order to deal with the mentioned problems, we introduce a classification framework with 
adaptive smoothing to remove small image features whilst protecting the discriminative structural 
information (i.e., edges). The appearance of the retained edges is enhanced using morphological dilation. The 
rest of this paper is organized: section 2 details out the data collection and pre-processing phase, section 3 
outlines the proposed method. The results and analysis are presented in section 4 whilst the conclusion is 
provided in section 5. 


2. DATA COLLECTION AND PRE-PROCESSING 

Red, green and blue (RGB) images of the decorative plaited mat motifs were captured indoor using 
a setup consisting of a downward-facing camera with normal setting. For this initial work of ours, only 20 
motifs were used in the experiments out of the possible 50. We labelled the class by the most prominent 
motif visible on the plaited mat upon consultation with the experts. The images were resized to a resolution 
of 800x533 pixels (original resolution is 5,184x3,456 pixels) for performance reason. The resized RGB 
images were then cropped and perspective-corrected to produce synthetic images. Following the decision 
made during our previous work [4] synthetic images are used as the target image instead of the actual 
photographed images to prevent background noises from polluting the resulting image descriptor. Examples 
of synthetic plaited mat motif images of different classes are shown in Figure 2. 

Common image deformations [25] of incremental degrees were applied to the synthetic images to 
produce our query sets. We used the same set of incremental degree values as in our previous work [4]. Since 
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we have a total of 20 classes, the precision of random classification by chance is 3 0-05. 


Iban plaited mat motif classification with adaptive smoothing (Silvia Joseph) 


842 im) ISSN: 2252-8938 


shih. 


Figure 2. Examples of synthetic images, 1.e., Buah Lang Antu, Buah Burung Garuda, Buah Bandau, Buah 
Lintan Tanah and Buah Nabau Besundang (from left to right) 


3. PROPOSED METHOD 

The proposed method utilizes adaptive smoothing and morphological dilation to not only remove 
noises but also to recover lost image information and to enhance image details. We start by convolving the 
800x533 greyscale image with a Gaussian kernel of size 5x5, with the 1“ derivation of 2D Gaussian 
smoothing filter function. The purpose of this smoothing function is to destroy the small image features that 
we considered as noise. The smoothed image is then converted into an edge image using Canny filter. In our 
proposed classification framework with adaptive smoothing, we gradually reduce the number of remaining 
edges by iteratively applying Canny filter of increasing low threshold value to the original edge image. We 
use a fixed increment of 5 units. By setting a higher value for the Canny filter’s low threshold parameter, the 
number of kept edges is lower as the algorithm becomes more stringent in rejecting weak candidates. We 
stop the iteration once the percentage of remaining edges (PoRE) is lower or equal to the set threshold values 
(i.e., 100.0%, 90.0%, 85.0%, 80.0%, 75.0%, 70.0%, 50.0%, and 20.0%). These threshold values are selected 
randomly as it is possible to tune the threshold and locate the optimal value directly. These thresholds are 
evaluated using a validating set. PoRE is calculated using, 


Current number of detected edges using PHT 


PoRE = x 100 (1) 


Initial number of detected edges using PHT 


We calculate the current percentage at each iteration using probabilistic hough transform (PHT) to 
extract the image’s structural information. The iteration will stop once PoRE is equal or less than the target 
value, as shown in Figure 3 for a flowchart of the adaptive smoothing function. This function is implemented 
inside our classification framework. 


Input Image 
low =5 
target = e.g,75.0 
percentage=100.0 
1 


percentage <= target percentage = (#current_lines /#start_lines) 
I++ 


#current_lines = HoughLinesP( ) 
low = low +5 


Figure 3. Our classification framework with adaptive smoothing 
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To validate our classification framework, we conducted an experiment for motif image classification 
against the following common image deformations, i.e., image blur, illumination changes, JPEG 
compression, viewpoint changes, scale+rotation changes, and scale changes. Using our dataset, we prepared a 
test sequence (i.e., query images) for each motif class. A test sequence contains five images. Each image 
exhibits a gradual increment of geometric and/or photometric transformation of the initial image. Following 
the recommendation by [25], we used the same 6 image deformation types as in our previous work [4]. For 
each motif class, both test image (i.e., synthetic image) and the test sequence’s images were converted into 
edge images at different PoRE values. The final edge images were used during the classification exercise. As 
a comparison set, we also produced a second set by applying PHT on the final edge images. 

In total, we have 20 motif classes x 6 image deformation types x 5 deformation levels x 8 sets of 
PoRE values range of (100%, 90%, 85%, 80%, 75%, 70%, 50%, and 20%), yielding a final total of 9,600 
query images. Figure 4(a) shows examples of query image produced at different PORE values, whilst Figure 
4(b) shows the corresponding query images with dilation (to improve appearance of edges). Figure 5 shows 
selected query images with incremental degrees of image deformation. For the feature matching and 
classification of the prepared target and query images, binary robust invariant scalable keypoints (BRISK) 
[26] were used. BRISK detects corners using adaptive and generic accelerated segment test (AGAST) 
algorithm [27] and perform filtering with features from accelerated segment test (FAST) corner score while 
searching for maxima inside the scale space pyramid. Outlier-rejection from matched features was performed 
using random sample consensus (RANSAC). RANSAC estimates the homography matrix and use that 
information to eliminate any mismatches/possible false key points. We used the ready implementation of 
BRISK and RANSAC in VLFeat Version 0.9.21 library [28]. This work was implemented on a Windows 
laptop with the following specifications: Intel Core i7-8550U @ 1.80GHz processor and 8 GB RAM. 
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Figure 4. From left to right, query images produced using PoRE values of 100.0%, 75.0%, and 20.0% for the 
following motif classes (top to bottom row) Buah Lang Antu, Buah Burung Garuda, Buah Bandau, Buah 
Tungku Lebur and Buah Nabau Besundang. Two versions are shown, i.e., (a) without dilation and 
(b) with dilation 
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Figure 5. From top to bottom, selected query images with incremental degrees of image deformation. From 
left to right, a column showing query images of Buah Lang Antu sequence for blur changes, Buah Burung 
Garuda sequence for JPEG compression changes, Buah Bandau sequence for illumination changes, Buah 
Tungku Lebur sequence sequence for viewpoint changes, Buah Nabau Besundang for scale changes, and 

Buah Tangkai Gajai Besembah sequence for scale + rotation changes 


4. RESULTS AND ANALYSIS 

We recorded the number of detected BRISK key points inside the query vs. template image, the 
number of correspondences, and number of correct matches after RANSAC. For image blurring, when the 
level of distortion increases, the number of detected BRISK+RANSAC key points decreases dramatically 
especially at the 5" level of deformation without dilation, shows in Table 1. However, with dilation enabled 
of PoRE=75.0%, the decrease in number does not correlate with a loss of accuracy. False matches only 
started at the 3 deformation level for image blurring, while for scale+rotation, we observed that the 
BRISK+RANSAC is able to detect a considerable number of key points, even at increasing degrees of 
deformation. We also found that that our method performed well for viewpoint with front parallel surface 
images at a value of 10° (left/right) and slanted surface images, at a value ranging from 20° to 60°. 
BRISK+RANSAC managed to detect enough key points, even at the worst deformation level of 
scale+rotation changes. Clearly it is demonstrated that BRISK+RANSAC using our proposed method 
performed well against JPEG compression, viewpoint and illumination changes, shows in Table 2. The 
unique properties of symmetrical, repetitive, similar and complex motifs, i.e., Buah Tangkai Gajai 
Bersembah and Buah Tungku Lebur creates a challenge for the classification task that leads to some errors 
during the matching process as shown in Figures 6(a) and 6(b). Motifs with simpler features, i.e., Buah Mata 
Punai that sometimes act as a filler for other motifs, have a smaller number of strong edges resulting to a 
reduced number of discovered key points, shows in Figure 6(c). 
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(c) 
Figure 6. Three motifs with high misclassification rate (a) Buah Bulan Bekarung, 


(b) Buah Tangkai Gajai Besembah, and (c) Buah Mata Punai 


Table 1. Number of detected BRISK+RANSAC keypoints using the optimal PoRE value of 75.0% for each 
image deformation category, with and without morphological dilation 


Image deformation w/o dilation w/ dilation 
I“level 5 level “level 5 level 

Blur 155 0 7882 164 
Illumination 117275 50991 140871 64996 
Viewpoint 457 36 2994 231 
JPEG compression 82992 1300 128084 10379 
Scale+rotation 397 30 1307 122 
Scale 795 138 2516 527 


The following metrics were calculated: 
a) True positives (TP): the number of query images being correctly classified with a similarity score higher 
than the threshold. 
b) False positives (FP): the number of query images being incorrectly classified with a similarity score 
higher than the threshold. 
c) False negatives (FN): the number of instances where multiple top matches were returned, or the top 
matches similarity score is lower than the threshold. 


Table 2. Comparative failure point of using BRISK+RANSAC with optimal PoRE value of 75.0%, 
for each image deformation category. We show the failure point (a) w/o morphological dilation, and 
(b) w/ morphological dilation 


(a) 
Level of image deformation Blur Hlumination — JPEG Compression _ Viewpoint Scale+rotation Scale 
Imagel FN TP TP TP FN FN 
Image2 FN TP TP TP FN FN 
Image3 FN TP TP TP FN FN 
Image4 FN TP TP FN FN FN 
Image5 FN TP TP FN FN FN 
(b) 

Level of image deformation _ Blur Illumination JPEG Compression _ Viewpoint Scaletrotation Scale 
Imagel TP TP TP TP TP TP 
Image2 TP TP TP TP FN TP 
Image3 FN TP TP TP TP TP 
Image4 FN TP TP TP FN TP 
Image5 FN TP TP TP FN FN 


The results of keypoint matching from both query and template image were summarized in the form 
of confusion matrix as shown in Figures 7(a) to 7(f). We have 20 actual classes and 20+1 predicted classes. 
The additional predicted class, i.e., class unresolved, represent cases where our classification returns multiple 
top matches or a similarity score is lower than the threshold. We consider such cases as a FN to penalise the 
configuration. Based on the confusion matrix obtained from the classification results using our proposed 
framework with optimal PoRE value of 75.0% + dilation, for Blur set as in Figure 7(a) shows the highest 
number of unresolved classes, followed by scale set as in Figure 7(d), and scale+rotation set, in Figure 7(f). 
Most of the results in class unresolved for blur set are due to the false top matches produced by simple motif 
Buah Mata Punai and complex motifs of Buah Bulan Bekarung, Buah Kandung Nibung Berayah, Buah 
Sebayan and Buah Tangkai Gajai Besembah. From our experiment, we observed that blurring is the most 
destructive image deformation type (compared to the rest) as its application produces many false top 
matches. There is no class unresolved for JPEG compression set, illumination set, and viewpoint set, see 
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Figures 7(b) to 7(e). Figure 8(a) shows a synthetic image of Buah Tungku Lebur, and Figure 8(b) shows the 
resulting Canny edge image. At PoRE value of 75.0%, the visualized detected BRISK features after blurring 
at increasing levels are shown in Figures 8(c) to 8(f). Side-by-side comparisons of visualized matching 
results for selected motifs using BRISK+ RANSAC at deformation level 1 (left column) vs. level 5 (right 
column) are shown in Figure 9. We include a sample visualized matching result each for the following image 
deformation types, i.e., i) blur, see Figure 9(a); 11) viewpoint, see Figure 9(b); iii) scale, see Figure 9(c); and 
iv) scale+rotation, see Figure 9(d). Clearly demonstrated that with our proposed method we managed to 
match enough key points, even at the highest deformation level. Based on the confusion matrix, accuracy is 


calculated as, 


TP+TN 
P 


(2) 


Accuracy = 


Where; TP= true positive, TN= true negative (by default is 0), P= population. 
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Figure 7. Confusion matrix obtained from the classification results using our proposed framework with 
optimal PoRE value of 75.0%-+dilation, for (a) blur set, (b) JPEG compression set, (c) illumination set, (d) 
scale set, (e) viewpoint set, (f) scalet+rotation set 
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Figure 8. Synthetic image of (a) Buah Tungku Lebur, (b) the resulting edge image produced by Canny filter, 
detected BRISK key points when using optimal PoRE value of 75.0% at (c) blur level 1, (d) level 2, and 
(e) level 3, and (f) level 5 


In Figure 10, we presented the average matching accuracy achieved by the following schemes: 
scheme 1: synthetic image and BRISK+RANSAC; scheme 2: synthetic image and BRISK+RANSAC with 
Canny filter; scheme 3: synthetic image and BRISK+RANSAC with Canny filter and PoRE (without 
dilation); and scheme 4: synthetic image and BRISK+RANSAC with Canny filter, PoRE (with dilation) the 
average accuracy is 85.0%, 95.0%, 79.0% and 96.0% respectively. As for dilation vs no dilation, we 
observed that with dilation enabled, the average accuracy increases by 17.0%. The optimal scheme, i.e., 
scheme 4 offers an average increase of 11.0% compared to the baseline, i.e., scheme 1, and an average 
increase of 1.0% compared to scheme 2. Figure 11, illustrate the average matching accuracy using 
BRISK+RANSAC at different PoRE values for each image deformation type. The optimal PoRE value is 
observed to be 75.0% with a comparably higher average matching accuracy of 83.0%, 100.0%, 100.0%, 
100.0%, 92.0% and 99.0% across all image deformations (blur, illumination, JPEG compression, viewpoint, 
scale+rotation and scale) compared to the rest. 


(a) 


(b) 


(c) 


(d) 


Figure 9. Side-by-side comparisons of visualized matching results of Buah Lang Antu, Buah Tungku Lebur, 
Buah Nabau Besundang and Buah Tangkai Gajai Besembah using BRISK+ RANSAC at deformation level 1 
(left column) vs. level 5 (right column). We show a sample matching result (optimal PORE=75.0%, dilation 
enabled) each for the following image deformation types, i.e., (a) blur, (b) viewpoint, (c) scale, and 
(d) scale+rotation 
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Performance comparison of average matching accuracy using BRISK+RANSAC at different 
scheme following six image deformations. 
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Figure 10. Performance comparison of average matching accuracy using BRISK+RANSAC at different 
schemes, following six image deformations 


Average matching accuracy of BRISK+RANSAC at different PoRE + dilation 
following six image deformations. 
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Figure 11. Average matching accuracy of BRISK+RANSAC at different percentage of PoRE+dilation values 
following six image deformations 


5. CONCLUSION 

In this work, we presented an adaptive smoothing scheme consisting of Canny edge detector and 
PHT to gradually remove smaller image structures from a given mat motif greyscale image. Dilation is then 
used to restore broken edges after the application of the smoothing scheme to improve detection. Motifs are 
matched using BRISK features and RANSAC against six common image deformations: combination of scale 
and rotation, viewpoint, image blur, JPEG compression, scale and illumination changes. Varying degrees of 
each deformation type were applied to all cropped and perspective-corrected mat motif images. The optimal 
PoRE value is 75.0% with a reported 100.0% matching accuracy for JPEG compression, illumination and 
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viewpoint change set. Our optimal scheme achieved an average increase in accuracy of 11.0% compared to 
classification using only BRISK+RANSAC on synthetic RGB images (i.e., baseline method). It is observed 
that the proposed scheme improves the classification accuracy of mat motif images with complex motif 
across common image deformation types. 
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